Learned digital lens enabled single optics achromatic imaging.

High-quality imaging with reduced optical complexity has been extensively investigated owing to its promising future in academic and industrial research. However, the practical performance of most imaging systems has encountered a bottleneck posed by optics rather than electronics. Here, we propose a digital lens (DL) to compensate for the chromatic aberration induced by physical optical elements, while the residual wavelength-independent degradation is tackled through a self-designed neural network. By transforming physical aberration correction to an algorithm-based computational imaging task, the proposed DL enables our framework to reduce optical complexity and achieve achromatic imaging in the analog domain. Real experiments have been conducted with an off-the-shelf single lens and recovered images show up to 14.62 dB higher peak signal-to-noise ratio (PSNR) than the original chromatic input. Furthermore, we run a comprehensive ablation study to highlight the contribution of embedding the proposed DL, which shows a 4.83 dB PSNR improvement compared with the methods without DL. Technically, the proposed method can be an alternative for future applications that require both simple optics and high-fidelity visualization.

Large depth-of-field computational imaging with multi-spectral and dual-aperture optics.

Large DOF (depth-of-field) with high SNR (signal-noise-ratio) imaging is a crucial technique for applications from security monitoring to medical diagnostics. However, traditional optical design for large DOF requires a reduction in aperture size, and hence with a decrease in light throughput and SNR. In this paper, we report a computational imaging system integrating dual-aperture optics with a physics-informed dual-encoder neural network to realize prominent DOF extension. Boosted by human vision mechanism and optical imaging law, the dual-aperture imaging system is consisted of a small-aperture NIR camera to provide sharp edge and a large-aperture VIS camera to provide faithful color. To solve the imaging inverse problem in NIR-VIS fusion with different apertures, a specific network with parallel double encoders and the multi-scale fusion module is proposed to adaptively extract and learn the useful features, which contributes to preventing color deviation while preserving delicate scene textures. The proposed imaging framework is flexible and can be designed in different protos with varied optical elements for different applications. We provide theory for system design, demonstrate a prototype device, establish a real-scene dataset containing 3000 images, perform elaborate ablation studies and conduct peer comparative experiments. The experimental results demonstrate that our method effectively produces high-fidelity with larger DOF range than input raw images about 3 times. Without complex optical design and strict practical limitations, this novel, intelligent and integratable system is promising for variable vision applications such as smartphone photography, computational measurement, and medical imaging.

Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning.

Hyperspectral imaging is being extensively investigated owing to its promising future in critical applications such as medical diagnostics, sensing, and surveillance. However, current techniques are complex with multiple alignment-sensitive components and spatiospectral parameters predetermined by manufacturers. In this paper, we demonstrate an end-to-end snapshot hyperspectral imaging technique and build a physics-informed dual attention neural network with multimodal learning. By modeling the 3D spectral cube reconstruction procedure and solving that compressive-imaging inverse problem, the hyperspectral volume can be directly recovered from only one scene RGB image. Spectra features and camera spectral sensitivity are jointly leveraged to retrieve the multiplexed spatiospectral correlations and realize hyperspectral imaging. With the help of integrated attention mechanism, useful information supplied by disparate modal components is adaptively learned and aggregated to make our network flexible for variable imaging systems. Results show that the proposed method is ultra-faster than the traditional scanning method, and 3.4 times more precise than the existing hyperspectral imaging convolutional neural network. We provide theory for network design, demonstrate training process, and present experimental results with high accuracy. Without bulky benchtop setups and strict experimental limitations, this simple and effective method offers great potential for future spectral imaging applications such as pathological digital stain, computational imaging and virtual/augmented reality display, etc.

End-to-end computational optics with a singlet lens for large depth-of-field imaging.

Large depth-of-field (DOF) imaging with a high resolution is useful for applications ranging from robot vision to bio-imaging. However, it is challenging to construct an optical system with both a high resolution and large DOF. The common solution is to design relatively complex optical systems, but the setup of such systems is often bulky and expensive. In this paper, we propose a novel, compact, and low-cost method for large-DOF imaging. The core concept is to (1) design an aspherical lens with a depth-invariant point spread function to enable uniform image blurring over the whole depth range and (2) construct a deep learning network to reconstruct images with high fidelity computationally. The raw images captured by the aspherical lens are deblurred by the trained network, which enables large-DOF imaging at a smaller F number. Experimental results demonstrate that our end-to-end computational imager can achieve enhanced imaging performance. It can reduce loss by up to 46.5% compared to inherited raw images. With the capabilities of high-resolution and large-DOF imaging, the proposed method is promising for applications such as microscopic pathological diagnosis, virtual/augmented reality displays, and smartphone photography.